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ABSTRACT 

Reading achievement of elementary school children in 
California has been measured by several separate studies, not all of 
which appear to present compatible results. This study of three 
assessments contained in the Miller-Unruh Report analyzes their 
purposes, methods, results, and recommendations in an effort to 
produce some way to arrive at an accurate picture of the reading 
achievement level in the state. The analyses of assessment studies 
indicated that inadequacies in methods, discrepancies in data 
collection, and differences in reporting techniques were largely 
responsible fcr the various results. A number of recommendations are 
made for application in future assessments and for improving reading 
instruction. These were directed at greater teacher preparedness, 
fuller understandings of what ought to be measured, more 
comprehensive and more uniform reporting of subtest scores as well as 
total test scores across grade levels and school districts, and 
increased financial support for experimental programs. Tables and 
references are included. (MS) 
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READ ING ACHIEVEMENT IN CALIFORNIA: MIRACLE OR MIRAGE ? 



"The Right to Read" is a phrase now frequently quoted both at 
national and state levels. The use of this statement is undoubted- 
ly prompted by the widely quoted statistic that one out of every 
four students has significant difficulty with reading and up to 
one-half of the pupils in our large city school systems read below 
their age and grade level.'*' The use of this phrase has also be si* 
used for political purposes by those aspiring to high public office. 

Individuals in the State of California have long been concerned 
with reading achievement. This is evident in the recent efforts 
resulting from such legislation as the Miller--Unruh Act, providing 
for special instructional assistance for youngsters encountering 
reading difficulty at the primary grade level, and also providing 
for mandatory reading achievement testing for all children in the 
State of California. Further concern about reading and language 
education in California is obvious from hearings such as those 
held by the California Assembly Committee on Education. 

However, the questions one must ask are what progress has 
been made in reading achievement by elementary school children in 
California during the past several years and what is the prognosis 
for the future? In order to answer these two key questions it is 
necessary to respond to five related questions which serve to outline 
the remainder of this discussion: 



The Role of the Teacher and the Right to Read , Leadership 
Training Institute, Swartluuore College, Swarthmore, Pa., 1970, p. 
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1. What is the purpose of reading achievement testing? 

2. What do the reading test results indicate? 

3. What does the test reporting fail to indicate? 

4. What factors deserve careful consideration in improving 
the reading achievement of California's primary grade 
children? 

5. What recommendations may be offered for more effective 
utilization of state and local resources in enhancing the 
reading instruction of California's youth? 

Purpose of Reading Achievement Testing 



According to Dr. James Crandall, one of the basic purposes of 
reading achievement testing and of the reports put out by the State 
Department of Education is as follows: 

"The reports have been made so that citizens and educators 
woujl have information about pupil achievement to use in 
strengthening the California public school system. 

In the same preface Crandall also states that: 

"The report is intended to provide information that the 
State Board of Education may find useful in making decisions 
about statewide educational program priorities, and that 
the Legislature may find useful in making decisions about 
allocation of available state resources. It is also in- 
tended to supply information that local school districts 
may use in strengthening existing curriculum and making de- ^ 
cisions about the development of new programs for children." 



o 

Dr. James H. Crandall, Chief, Bureau of Evaluation and In- 
structional Research, CALIFORNIA STATE TESTING PROGRAMS 1968-69. 
An Analysis of Reading Test Scores and Other School Factors. 

3 Ibid. 
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Dr. Crandall’s initial comments certainly lead one to expect 
that such information would be of value to a wide range of persons 
to legislators who need definitive information for decisions on 
funding; to state and local administrators as assessment tools for 
achievement and growth in individual school districts; to teachers 
needing to assess classroom pupil growth; and to parents wishing 
to know how their children were progressing in school. 



What Do the Reading Results Indicate ? 



The first document examined was the Shanner Report. At face 
value, this report establishes the impression that children in 
California are making tremendous strides in their scores on read- 
ing achievement tests. ^ 

A major measurement problem in Dr. Shanner 1 s analysis is the 
fact that a child who simply guessed at the answers in a multiple 
choice test would produce scores on that test far above the arbi- 
trarily selected lower raw score* criterion reference point. For 
example, by combining two sub- tests (which was the way in which 
Shanner received the data), the total Stanford Achievement Test, 
Primary I, has 73 items. The test is multiple choice with four 
possible choices in each case. Since 25 percent (1 out of 4) 
times 73 = 18.25, we can assume that even if a child were not able 
to read, he could guess correctly on approximately 18 items. Dr. 
Shanner’ s report cites 75,866 children who scored 20 and below, 
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Report from Dr. W. M. Shanner (April 8, 1970) to Dr. James 
Crandall. Subject: Review of Stanford Reading Test Results, Cali 

fornia State Testing Program, 1966-69. 
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and one could assume (since the number of children is so large) 
that those in the 20 and below category and those in the 10 and 
below category were probably unable to read the test in the first 
place . 

Dr. Shanner’ s next analysis was on the Stanford Reading Achieve- 
ment Test Primary II. Again, this was a multiple choice test with 
four possible choices for each item. However, this total test, 
which again combined two sub- tests, contains 96 items. Thus by 
the same reasoning (25% x 96) one could assume that a child who 
was unable to read would guess correctly 24 times out of 96, Dr. 
Shanner cites 56,447 children who scored 20 or below and a total 
of 123,116 children who scored 30 and below. Thus the number of 
children who could pass the test purely by guessing at the arbi- 
trarily established level is somewhere in the neighborhood of 
80,000 to 90,000. 

A third feature of the data which is hazy is that some chil- 
dren are able to do part of the test and then guess at the remain- 
der. Thus there is undoubtedly some element of error in Shanner 1 s 
remaining criterion points even though it would not be possible to 
decipher the exact degree. 

One of Shanner 1 s major conclusions contained on page 3 of his 
April 8, 1970, cover letter to Crandall is as follows: 

M Thus, one is getting a 2.59%, growth or improvement in 
pupil performance on the average of each raw score inter- 
val in terms of student population. For the four-year 
interval 1966-69, the value is 5.03%,. This to me seems 
a satisfactory growth record." 

This latter statement by Shanner falls into the category of 
value judgments since what one person feels is satisfactory growth 
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may appear to a second person to be unsatisfactory growth. How- 
ever, even if we consider Dr. Shanner ! s 5% calculated improvement 
in 4 years as an accurate figure, the question then becomes whether 
or not this is an extraordinary rate of growth in reading achievement. 

To put this in terms of the man on the street, i.e., the me- 
chanic, the salesman, or the assembly line worker in a factory, we 
might by way of analogy wonder if he or she would think it extra- 
ordinary to receive a 5% pay increase after working for 4 years. 

For example, a person earning $500.00 per month would receive a 
$25.00 a month raise after four years of work . Thus it seems 
essential to distinguish between two concepts: that of statisti - 

cal significance and that of practical significance . 

Perhaps a second analogy would clarify this point more fully. 
When one is dealing with very large numbers (a large sample popu- 
lation) differences which are almost microscopic in nature could 
be called statistically significant . For example, if a million 
men were inducted into the armed forces, and they were all measured 
very carefully, the average height might be 5 feet 9-% inches. Thus 
a man who was 5 feet 9-3/4 inches could certainly claim that he 
was significantly taller than average. However, even if two men 
(5 feet 9-^ and 5 feet: 9-3/4) stood side by side it would probably 
be impossible to distinguish between them. 

A second document examined, the Analysis of the Budget (Sum- 
mary of State Expenditure for Education) Miller-Unruh Basic Read- 
ing Scores, clearly indicates that one or two point increases in 
raw scores often do not result in any increase whatsoever in grade 
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equivalency scores. This results from the fact that it often re- 
quires an increase of 2 or 3 points in raw scores to be the equiva- 
lent of 0. 1 of a year in grad e equivalent scores . For examp le, 
the scores on the following table have been extracted from the above 
noted document. 

In light of the negligible grade equivalency gains shown in 
the bottom half of the following table, it is difficult to recon- 
cile the actual data with Dr. Max Rafferty's recent statement (News- 
maker, Channel 5, August 21, 1970) that the reading achievement 
gains of California children are "probably the most significant 
thing that has happened in California education in the past 20 
years . 11 

If one looks closely at the data, it appears quite obvious that 
it will take California children approximately 12 years to reach 
the publisher's norms in reading achievement. In addition, this 
in itself is hypothecated on the presumption that the present up- 
ward trend continues at the present rate. Increased financial 
support and increased allocations for Miller-Unruh teachers and 
specialists in teaching reading could undoubtedly cut this time 
lag considerably and thus, regardless of political outcomes or 
temporary economic declines, we would hope that such support would 
be forthcoming. 

A third document is the Miller-Unruh Report- -a document which 
appears to be large and unwieldy but which contains a great deal 
of valuable information which has been analyzed and presented in 
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CALIFORNIA RAW SCORES ON THE STANFORD ACHIEVEMENT TEST 
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Note that no increases in grade equivalency occur for children in the lower quartile. The few .1 year 
gains occur only for those youngsters at the median and upper quart ile. 
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an easily readable and statistically meaningful manner. ~ The Miller- 
Unruh Act has quite clearly kept poor readers from falling further 
and further behind and the report itself states that even though 
the Miller-Unruh schools (those with a Miller-Unruh reading spe- 
cialist) have not kept pace with California children as a whole, 
that caution should be used in interpreting these data because the 
"children identified for direct assistance from a specialist teacher 
have been selected for such assistance because they are poor readers." 

What Does the Test Reporting Fail to Indicate ? 

The value of testing information is, of course, directly re- 
lated to the form in which data are classified, organized, and 
interpreted. The Crandall Report rank order s each school district 
on a series of categories such as the (1) index of family poverty, 

(2) assessed valuation per unit of average daily attendance, and 

(3) expenditure for instruction per average daily attendance. ^ 

The problem, however--and the key ingredient making the data 
very difficult to use as a basis for decision-making- -is that each 
category has a different number of ranks . In examining the first 

5 MILLER-UNRUH BASIC READING ACT OF 1965 program report for 
1966-1970, and Analysis of Test Scores for May 1968 and May 1969. 

£ 

All available research points toward the fact that poor 
readers fall further and further behind in each progressive year. 

Thus, by preventing this typical typ e of regression the Miller- 
Unruh program has indeed performed a valuable service. (See 
Appendix.) 

^Dr. James H. Crandall, 0 £. cit . 
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entry on page one (Alameda City Unified), the reader must do a 
series of calculations to find out where that particular district 
actually fell in the various rankings and then calculate literally 
thousands of figures to determine where Alameda City Unified fell 
in relation to any other school district. In total there are 17 
columns containing rankings and Alameda* s entries are as follows: 

116 out of 301, 71 out of 144, 93 out of 232, 78 out of 160, 437 

out of 804, 164 out of 422, 5 out of 52, 8 out of 32, 817 out of 

1076, 265 out of 1061, 20 out of 257, 49 out of 381, 17 out of 76, 

81 out of 162, 84 out of 197, 332 out of 419, and 3 out of 49. 

Thus, as the data now stand, the appearance is simply that of an 
IBM print-out sheet which should be subjected to analysis rather 
than anything even resembling an analysis in itself. 

The most logical approach--and certainly the one most usable 
not only for the layman but also for educators, researchers and 
others concerned with education--would be to convert the data into 
equivalent rankings with one (1) being high and one hundred (100) 
being low. This is simply a question of running the card punched 
data through the proper computer program, and presumbaly the data 

Q 

are already in card punched form. 

In looking at the far right-hand corner of page one of the 
Crandall Report, the reader will note the heading Median Reading 
Test Raw Scores , and this in turn is broken down into grades one, 

Q 

°The reason for some of the categories having such large ranks 
such as 1076 and 1061 is that the analyst apparently felt it essen- 
tial to calculate the data to the closest cent rather than the closest 
ten cents or the closest dollar. 

10 
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two, and three. However, no indication is given that the grade 
one test actually contains 73 items whereas the tests for grades 
two and three contain 96 items. This could easily mislead the 
reader into thinking that the raw scores are comparable from grade 
to grade and even if he were not misled, he would have to determine 
where his district fell in relation to any other. Needless to say, 
the data could easily have been converted into percentages (by 
IBM equipment) but as they now stand they are totally unusable. 

A third major problem in using the Craudall Report is that 
two separate tests contained in the Stanford Achievement Test have 
been grouped together. The test as given in the classroom is broken 
down into Word Meaning which gives an indication of the child T s 
word knowledge, iid Paragraph Meaning which gives an indication of 
the child 1 s ability to comprehend the meaning of entire paragraphs. 
However, as a result of combining these two tests into one score, 
a school district would be unable to decipher whether or not the 
major problems revolved around Word Meaning or Paragraph Meaning. 

To some degree the foregoing criticism about combining scores 
may appear to be water over the dam since California is shifting 
to the Cooperative Primary Test (ETS) . However, there are three 
distinct reading-related parts of the Cooperative Primary Test 
which are separately scored and it is highly recommended that 
each part be made mandatory throughout the state and be reported 
as separate scores. In the Cooperative Primary Test these three 
items consist of: 

1. Reading- -which encompasses word meaning, sentence meaning 



0 



and paragraph meaning comprehension; 
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2. Word Analysis - -which centers on phonics; and 

3. Listening - -which enables one to obtain an indication of 
the child's word meaning, sentence meaning and paragraph 
meaning comprehension abilities without requiring the child 
to use word attack skills, e.g., phonics, to read words. 



Factors Leading to Improvement of Reading Achievement 

Reading is an extremely complex process. Successful reading 
is not a simple matter of having a child subscribe to a "phonics" 
or "whole word" approach. And in point of fact such a simplistic 
viewpoint is refuted by the massive twenty-year targeted research 
and development effort sponsored by the U.S. Office of Education. 

The basic objective of the above-noted project is to define the 
nature of the reading process and to develop programs leading to 

9 

successful reading achievement. 

Although there is a significant body of research to support 
the value of a strong word attack emphasis in reading instruction, 
this represents only one essential component of a successful reading 
program. ^ Certainly the importance of building comprehension 



Q 

William J. Gephart, "The Targeted Research and Development 
Program on Reading," Reading Research Quarterly , Vol. V, No. 4, 

Summer 1970, pp. 505-532. 

^Russell G. Stauffer (editor). The First Grade Reading Studies : 
Findings of Individual Investigations, International Reading Asso- 
ciation, Newark, 1967. 

Guy L. Bond and Robert Dykstra, Coordinating Center for First - 
Grade Reading Instruction Programs , U.S. Dept, of Health, Education, 
and Welfare, Final Report, Project No. X-001, Contract No. 0E-5-10-264. 
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abilities cannot be underestimated. In addition, positive atti - 
tudes toward the reading act are of great importance if the reader 
is to use his reading skills to any advantage. The recent adoption 
of state reading texts offers a range of emphasis on word attack 
skills from the Bank Street material, which resembles the previous 
state adoption in word attack pacing (Allyn-Bacon, Ginn) through 
Harper-Row and Macmillan to the Lippincott series, which increases 
the pacing of word attack skill development, e.g., phonics. There 
is also an emphasis on comprehension in each of these programs even 
though in many respects it is very similar to the previous state 
adopted programs. The use of supplementary materials such as the 
Holt, Rinehart and Winston series (Sounds of Language) should produce 
more positive attitudes toward reading and language learning. 

Despite the fact that there is a range of reading programs 
available in California, it is important to,. understand that the 
very research which supports an emphasis on word attack skill and 
comprehension development also indicates that more variation in 
reading achievement exists between teachers than between reading 
programs. ^ In other words, the critical agent of change in read- 
ing instruction is not only found in reading programs possessing 
certain instructional charac teristics--but more importantly in the 
teacher. This appropriately calls for a shift in emphasis from 
finding the "magic" text to preparing the efficient teacher, 

•^Stauffer, Bond and Dykstra, 0£. cit . 
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In addition to the foregoing, we must carefully examine the 
limited time and support given to the preparation of reading language 
teachers at the college and university level. The thirty clock 
hours devoted to preservice teacher training in reading at the 
University of California at Berkeley, for example, provides time 
to do little more than overview fundamentals. Such programs not 
only need larger blocks of instructional time but also need to 
consider undergraduate course preparation in reading related dis- 
ciplines (see later comments on Miller-Unruh teacher preparation). 
Programs of reading instruction at the college and university level 
must also reexamine the location of the instructional setting and 
move outward into the community to teach reading methods courses 
in the schools where direct contact with children can be frequently 
developed . 

Inservice training programs in reading and language instruc- 
tion in school districts also need to be reexamined with support 
from state department personnel. Teachers are often presented 
with inservice training which may be described at best as a "shot- 
gun” approach. Consistent and ongoing teacher training centers 
in reading instruction are rare and must be established in school 
districts. These centers must be staffed with highly skilled pro- 
fessionals who plan for intensive inservice work in reading and 
language instruction. 

Special inservice programs are essential for the Miller-Unruh 
reading teachers if the stated objectives of "prevention of reading 
disabilities and the correction of reading disabilities at the 
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earliest possible time in the educational career of the pupil" 
(C alifornia Education Code, Section 5771) are to be achieved. The 
present and frequently used option of recommending a Miller-Unruh 
teacher at the local school level and then certifying the teacher 
on the basis of a written examination on reading instruction is 
at best an ineffective approach to developing highly competent 
reading specialists. The present system of granting credentials 
for the Miller-Unruh teachers must be reexamined and provision 
made for formal training as a reading specialist. Such training 
should include the following: 

1. Formal work in diagnosis and remediation of reading dif- 
ficulties; 

2. Instruction in childrens literature; 

3. Study of basic language concepts, including implications 
for understanding childrens reading and language growth 
and development (linguistics, psycholinguistics, etc.); 

4. Study of the relationship between the oral language sys- 
tem and the written language system of English (linguistics); 

5. Examination of syntactical elements which produce distinct 
meaning changes within and across sentences (linguistics, 
psycholinguistics) ; 

6. Study of concept development and thinking strategies (psy- 
chology, psycholinguistics) ; 

7. Study of socio-ethnic variables and school-community re- 
lationships (sociology, social psychology, sociolinguistics); 

8. Study and understanding of dialect differences which will 
be of significant value in reading and language instruction 

O 
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for teachers and children (particularly low socio-economic 
Black, white and Chicano youngsters) speaking standard 

1 9 

and nonstandard English respectively (sociolinguistics). 

It is imperative that as much information as possible be avail- 
able to the reading teacher in the classroom. By providing evalua- 
tion instruments which assess word meaning, paragraph meaning (read- 
ing and listening comprehension) and word attack skills the nature 
of the child's reading difficulty can more easily be identified. 

For example, a low score on paragraph meaning may indicate that 
the child is weak in comprehending language. On the other hand, 
it may merely indicate that the child is unable to pronounce the 
printed words in order to utilize his comprehension ability. Thus, 
by utilizing information on comprehension and word attack the teacher 
can more readily discern the nature of the child's problem, using 
this information in conjunction with his own individual diagnostic 
techniques to focus on the specific nature of the child's problem 
in comprehension or word attack skill. In this manner the state 
achievement testing program can serve as a group screening device 
valuable to classroom instruction. Such information would also be 
of value at the state and local administrative level in more accurate- 
ly assessing the value of reading programs. 



It is not unusual, for example, to find that a teacher has 
diagnosed the oral reading of a nonstandard speaking Black youngster 
as poor oral reading when in fact the youngster is translating con - 
sistently and c orrectly in his own dialect . 




-16- 



Recommendations for Enhancing Reading Achievement of 
California 1 s Primary School Children 

Although the maximum raw score increase of one to three points 
which occurred in several instances represents a statistically 
significant increase in reading achievement, this increase repre- 
sents only a 0.1 year grade equivalency gain. This occurred in 
five instances in grades one through three and was always at the 
fiftieth and seventy-f if th percentile.. Thus the great majority 
of raw score increases, although statistically significant, are 
of little practical significance in terms of meaningful reading 
growth. A school administrator or curriculum director, for example, 
would be extremely reluctant to expend large sums of money on a 
totally different reading program in order to increase reading 
achievement by 0.1 of a year even if this occurred at all levels, 
which was not the case in the California test data. 

The job of developing educational programs leading to achieve- 
ment gains is an extremely difficult effort. Thus it is of para- 
mount importance that support be marshalled at federal, state and 
local levels if progress in reading achievement is to be made. 

The nature of this support is suggested below. 

The following recommendations are based on the foregoing 
analysis of the documents described: 

1. That the statewide testing programs report separate test 
scores for each sub-test on reading comprehension, word 
meaning, word study skills, and listening. This should 
facilitate the interpretation and use of testing information 
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2 . 



3 . 



4 . 



5 . 



not only in program planning at the state level, but it 
should also aid the classroom teacher in using test results 
for instructional purposes within the classroom by identi- 
fying strong and weak areas of the reading program, i.e., 
comprehension and word attack skills. 

That the test scores be reported not only in raw scores 
but also in terms of grade equivalents. The latter re- 
porting technique would be much more easily interpreted 
by the average citi 2 ,en, and it appears to be essential 
since ease of interpretation is stated to be a major ob- 
jective of the state testing program (see earlier quote 
from Dr* Crandall's report). 

That test results be reported in such a manner that varia- 
tions on such important factors as socio-economic status, 
intelligence, and beginning reading level are broken out 
separately in order to make valid comparisons between 
comparable schools. 

That any rank order type of data reporting be standardized 
on a l-to-100 scale so that it may be easily comprehended 
by the typical reader. 

That the State Department of Education focus its emphasis 
on the teacher and on inservice teacher training programs 
with less emphasis on text materials; and that the state 
provide consistent and continuing emphasis on inservice 
training of reading teachers. (College and university 
departments should also be encouraged to provide leader- 
ship in this effort.) 
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6. That colleges and universities with the support of the 
State Department of Education focus its efforts in pre- 
paring teachers of reading and reading specialists by: 

a. Devoting more time to reading and language diagnosis 
and instruction; 

b. Moving methods courses into the public schools to 
provide the student teacher with children who may be 
used in instructional programs rather than having 
these courses remote from the actual primary grade 
classrooms ; 

c. Including related courses in the undergraduate prep- 
aration program for potential student teachers, e.g., 
linguistics, psychology, as related to special learn- 
ing problems of the culturally different child, and 
methodology of reading instruction. 

7. That Miller-Unruh reading teachers receive special prepara- 
tion as reading specialists. This should include courses 
in diagnosis and remediation of reading difficulties, and 
courses related to the unique problems of culturally 
different youth, e.g., non-standard dialects, concept 
development, attitudes toward learning, etc. This would 
best be accomplished by providing stipends to teachers 

for attending specialized summer and inservice courses, 

8. The Miller-Unruh state department staff should be expanded 
from two staff members to the number required for effec- 
tive operation in implementing the legislation. 
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9. That special model reading experimental schools be estab- 
lished to explore and formulate new teaching techniques 
and approaches in order to better match pupil learnf.ng 
styles with instructional approaches. Such a model school 
should recruit outstanding reading teachers and provide 
for detailed observation of the instructional approaches 
used. Such experimental schools should be established 
in areas of California representing a wide range or socio- 
economic and ethnic groups of primary grade children. 

10. That financial support be provided for: 

a. Meaningful test analysis and interpretation; 

b. State department leadership for inservice training 
of classroom teachers and Miller-Unruh teachers; 

c. Local district inservice training of classroom teachers 
and Miller-Unruh teachers, and expansion of state 
department Miller-Unruh personnel; 

d. College and university courses in reading instruction 
designed to support inservice teacher training for 
both classroom teachers and Miller-Unruh teachers; and 

e. Establishing model reading experimental schools to 
explore and formulate new reading techniques and approaches 
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APPENDIX* * 



READING ACHIEVEMENT SCORES USING THE STANFORD AND 
CALIFORNIA TESTS OF READING ACHIEVEMENT 



Number of Months Reading Age is Above 
or Below Chronological Age 
(Mean) 



Grade 


High Group 
(N=35) 


Low Group 
(N=35) 


Central Group ^ 


Total Group 


4 


+23.52 


-19.33 


+0.65 


+3.14 


5 


+32.22 


-20.08 


+4.24 


+5.64 


6 


+36.20 


-24.12 


44.37 


+5.45 


7 


+33.07 


-32.41 


+1.31 


+1.19 


8 


+25.57 


-32.63 


-0.63 


-1.08 



4 5 6 7 8 
Central Group N «= 145 149 142 122 101 
Total Group N = 188 206 198 177 141 



* Data calculated by Arthur Williams on the longitudinal research of 
Walter Loban. See unpublished report to U.S. Office of Education: 
Language Ability: Grades 10, 11 and 12 . August, 1967. 
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